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DETAILED ACTION 

Continued Examination Under 37 CFR 1. 1 14 

A request for continued examination under 37 CFR 1.114, including the 
fee set forth in 37 CFR 1 .17(e), was filed in this application after final rejection. 
Since this application is eligible for continued examination under 37 CFR 1.114, 
and the fee set forth in 37 CFR 1 .17(e) has been timely paid, the finality of the 
previous Office action has been withdrawn pursuant to 37 CFR 1.114. 
Applicant's submission filed on 2/12/2007 has been entered. 

Response to Arguments 

Applicant's arguments, see Remarks pages 1-6 and claim amendments, 
filed 2/12/2007, with respect to the rejection(s) of claim(s) 1-26 and 28-53 under 
35 USC 1 03(a) have been fully considered and are persuasive. 

Therefore, the rejection of claims 1 -26 and 28-53 has been withdrawn. 

However, upon further3 consideration, a new ground(s) of rejection 
against claims 1 -26 and 28-53 is made in view of various references as set forth 
below. 

The claim 1 as written does not exclude prior steps of rasterizing graphics 
data, receiving a user selection of a graphic that is then converted back to a 
location in a hierarchical data structure of the recited 'unrasterized base graphic 
data.' 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 
148 USPQ 459 (1966), that are applied for establishing a background for 
determining obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at 
issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

Claims 1-3, 21-22, 32, 44, and 52-53 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Kinoe (US 6,337,700) in view of Seidl (US 
5,710,896 A). 

As to claims 1 , 52, and 53, (method, system, computer-program product), 
Kinoe teaches: 

A computer-implemented method for highlighting a selected object on a display, 
the method comprising: 

-Receiving unrasterized base graphic data comprising a selected graphic object 
to be highlighted; (Kinoe displays the original image on CRT 22 or like display 
(e.g. 2:60-63), where the system then determines what (if any) graphical object 
was initially selected by the user (2:64-3:1 5). In order for that to take place, 4:45- 
65, 9:45-10:20, etc, the system determines what graphical object is selected, 
wherein such a graphic object is represented in a non-rasterized format, e.g. data 
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structure Figures 3-5, etc. Highlight types in 9:57-10:30 are highlighted, 
translucent, and/or translucent highlighted modes.) 

(a) Kinoe teaches: 

-Providing selection graphic data including a graphic object corresponding to the 
selected graphic object; (Kinoe clearly sets forth that the specific object to be 
modified or selected is provided (13:65-14:15, for example). The term "providing" 
does not require any particular functionality, merely the existence of the recited 
object.) 

(b) Kinoe does not expressly teach the counterpart graphical object, but Seidl 
clearly provides such a teaching: Seidl in Figure 3B shows several different 
buffer layers (8:25-67), wherein Seidl teaches that Seidl 19:64-20:5 clearly states 
the concept of having three buffers: one that represents the unselected objects, 
one that represents the selected objects, and a composite buffer that contains a 
graphical image of the entire display for redrawing changed portions of the entire 
display for redrawing changed portions of the entire display image) 

Kinoe and Seidl provide the concept of the unrasterized graphical object. 
Seidl makes clear that graphic components are graphic objects that are not 
rasterized but rather are data structures - see Figures 2 and 3, 2:15-20, where 
component objects, defined in 7:2-23, are held in the Model class, and are not 
rasterized unless they are on a Canvas class. The selected object - shown 
surrounded by a bounding box in Figure 3 - is therefore an actual data structure 
rather than raster data. Canvas class objects contain unrasterized data! 
Clearly 8:25-40 states, in part: "...The canvas class also maintains data 
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structures which support pluggable updates. In particular, it maintains three lists 
that partition all components in the model into equivalence classes. 1 . 
Background objects: all objects that are fully behind all objects currently selected. 
2. Foreground objects: all objects that are fully in front of all objects currently 
selected. 3. Midground consists of all components not in the foreground 
background. Fig. 3B illustrates the components in the graphic model in 
accordance with a preferred embodiment. The canvas class performs draw 
functions upon these objects (8:45-67) to actually generate the contents of the 
buffers per se. 

Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify the Kinoe reference to have an 
unrasterized version of the selection graphic object since both the Kinoe and 
Seidl references discuss the use of highlighting and compositing such objects 
and it would be much easier to manipulate and efficiently store the unrasterized 
version of the graphic object (e.g. graphic object data structure) of Seidl than a 
rasterized version thereof, since object-oriented programming provides data 
structures that are reusable and do not need to be recreated from scratch (e.g. 
the normal advantages of object-oriented programming, which Seidl extols (see 
Seidl 1 :58-2:10, 2:62-7:65, particularly the use of the Model class (Figs 2-3, 6:45- 
7:65, etc) 

Kinoe does teach rasterizing graphics, but does not expressly teach rasterizing a 
"base graphic" per se, specifically clearly teaching the limitations of rasterizing 
base graphic data, whereas Seidl teaches: 
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-Rasterizing the base graphic data to yield a base graphic raster; 

(Seidl clearly sends such graphic components to Canvas objects, which belong 

to the buffers shown in (19:64-20:5), where such are drawn as explained in 8:25- 

67) 

-Rasterizing the selection graphic data to yield a selection graphic raster; and 
(Seidl 19:64-20:5, Figures 3-5 clearly states the concept of having three buffers: 
one that represents the unselected objects, one that represents the selected 
objects, and a composite buffer that contains a graphical image of the entire 
display for redrawing changed portions of the entire display for redrawing 
changed portions of the entire display image, wherein the Seidl clearly teaches 
the composite graphic raster, which is the 'first data store' of Seidl) 
-Compositing the base graphic raster and the selection graphic raster to yield an 
output graphic raster for display, the output graphic raster including pixels 
representing a highlighting of the selected graphic object. (Seidl 7:25-9:15, 
19:64-20:5, Figures 3-5, etc, clearly teaches that the non-selected and selected 
graphical objects are combined such that composite graphics raster has the 
changed objects overwritten by those in the selected graphical object register, as 
explained above) 

Kinoe clearly teaches the above limitations of rasterizing base graphics 
data to provide a first initial view of the model, which is clearly stored in the frame 
buffer of the screen. Kinoe composites the highlighted object with the original 
base graphics raster so that it will be displayed on the screen, since the 
highlighted object is shown as being displayed. 
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Motivation to combine Seidl and Kinoe has already been provided above. 

As to claims 21 , firstly, Kinoe assigns highlighting values to graphical 
objects (see for example Fig. 4, the highlighting attribute 217 in graphical object 
state table 109), where a graphic object is prima facie composed of pixels, where 
clearly as set forth by Seidl the selected object would be in its own memory area 
(or selection raster) as set forth in the rejection to claim 1 as previously 
discussed, the rejection to which is incorporated by reference. 

That is, Kinoe teaches changing an attribute of an object (which would 
exist in the selection graphics raster) by assigning a highlight value to it. Seidl 
clearly teaches that when the object was updated the Canvas class would call 
the updater of that particular Component object within the model. Updating 
and/or drawing changes would proceed in the manner set forth in Seidl 8:25-65. 
The changed object would then be drawn, as set forth in 8:45-67. It would be 
obvious to perform this limitation in this manner, e.g. to modify Kinoe, for the 
reasons set forth in the rejection to claim 1 , which is incorporated by reference. 

As to claim 22, A method according to claim 21 wherein altering values of 
pixels from the base graphic raster comprises replacing the values of pixels from 
the base graphic raster with the highlighting values of corresponding pixels in the 
selection graphic raster. Seidl 8:25-67, when the updater accesses an object and 
sends the write command, it will obtain a copy of the object in the database to 
alter, and when the alterations are complete, the object in the buffer will be 
updated with the more current version. 
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As to claim 2, Kinoe does not expressly teach: wherein providing the 
selection graphic data comprises copying the selected graphic object from the 
base graphic data. Seidl clearly teaches that the data in the selected graphics 
buffer is copied from the object-oriented tree, SEE FIGURE 3, thusly teaching 
that the selected graphic object is copied from the base graphics data. 

As to claim 3, Kinoe teaches: wherein providing the selection graphic data 
comprises assigning a highlighting attribute to the copied selected graphic object. 
Kinoe clearly teaches this limitation in 2:52-60 and 3:1-28, where it is taught that 
objects are structured in a hierarchy on the display, and that such objects can 
have various attributes and uses the term 'translucent' to mean 'transparent' in 
certain contexts, for example in 9:57-1 0:30 it is clearly taught that the graphical 
objects can be in highlighted, translucent, and/or translucent highlighted modes. 
Clearly, this means that each object has an attribute to indicate whether or not it 
is highlighted. 

As to claim 44, it would be obvious that if Kinoe can turn on highlighting by 
changing the highlighting attribute in the state table of Fig. 4, that clearly Kinoe 
could turn off highlighting in the same manner. Obviously, the change would be 
made to the layer that the object was on, and then those raster layers would be 
composited to form the output image as recited in the claim. Motivation and 
combination is taken from the rejection to claim 1 . 

As to claim 32, A method according to claim 8 wherein providing the 
selection graphic data comprises assigning a blank attribute to the copied non- 
selected objects. Kinoe teaches very clearly that the "highlighting attribute" 217 
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in graphical object state table 109 in Fig. 4 has a state of '0' or T, that is, an 
object is either designated as highlighted ('1') or not ('0') - see 10:5-35. Objects 
by default are not highlighted (see for example Fig. 13). User selection and/or 
action result in highlighting attributes being enabled on specific parts (see Figs. 
1 4 and 1 5 for example). As stated in the rejection to claim 1 , which is 
incorporated by reference, the objects are copied in order to perform occlusion 
processing. Therefore, It would have been obvious to one of ordinary skill in the 
art at the time the invention was made that since objects by default are in a non- 
highlighted state, and that only the object that the user selects is highlighted, that 
such non-selected objects would have a blank attribute assigned to them, where 
the specification defines 'blank attribute' to mean 'non-highlighted.' 

Claims 4-16, 28, 49 are rejected under 35 U.S.C. 103(a) as 
unpatentable over Kinoe in view of Seidl as applied to claim 3 above, and 
further in view of Kang et al (US 6,266,068 B1). 

As to claim 4, 13, and 15, wherein rasterizing the selection graphic data to 
yield the selection graphic raster comprises assigning highlighting values only to 
pixels in the selection graphic raster corresponding to portions of the selected 
graphic object that are not overlapped by other non-transparent graphic objects. 
References Kinoe and Seidl do not expressly teach this limitation. However, it is 
well known in the art that three-dimensional graphics rendering systems such as 
Kinoe (15:47-60) do not display parts that are hidden from view or the portions 
that overlap thereof. 
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Kang et al (US 6,266,068 B1) clearly teaches the existence of multiple 
graphics planes in Figs. 2 and 3A-3E, and in 2:30-65, where as stated above, 
one of those layers could be the base graphics raster. Kang clearly teaches that 
users can select objects; see for example the intra-layer selector 106 in Fig. 1 as 
taught in 4:19-35. Kang teaches in 1 1 :37-63 that a user can highlight a layer 
using the input device to select the layer in question, with each layer having a 
different depth (e.g. see for example Figs. 3A-3E with emphasis on Figs. 3C and 
3D with the shaded ellipse having different depths in each, see for example 8:22- 
46). 

Kang further teaches that his system can deal with complex occlusion 
scenarios (1 :38-61), and this is noted in 4:20-35, where layers that occlude or are 
blended are disclosed, and in 7:10-30 the use of a painter's algorithm is 
disclosed, where this algorithm is well known in the art to ensure that objects in 
the scene occlude each other correctly based on depth, except where such 
objects are blended because they are partially transparent as set forth in 4:20-35 
for example. 

All that being said, it would be obvious to one of ordinary skill in the art at 
the time the invention was made that only the non-occluded portions of such an 
object would be drawn when a painter's algorithm or similar is used so that the 
scene is drawn in a realistic manner (excluding pixels that are blended as set 
forth above). Therefore, since only the portions of the selected graphical object 
that are not occluded will be drawn, (requires less system resources and is 
therefore inherently faster, more efficient, and requires less powerful hardware to 
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implement than its lack.) It would be obvious to only highlight the portions that 
are visible, as the rest of the object will not be drawn anyway. 

As to claims 5, 1 1 , and 16, firstly, Kinoe assigns highlighting values to 
graphical objects (see for example Fig. 4, the highlighting attribute 217 in 
graphical object state table 109), where a graphic object is prima facie composed 
of pixels, where clearly as set forth by Seidl the selected object would be in its 
own memory area (or selection raster) as set forth in the rejection to claim 1 as 
previously discussed, the rejection to which is incorporated by reference. 

That is, Kinoe teaches changing an attribute of an object (which would 
exist in the selection graphics raster) by assigning a highlight value to it. Seidl 
clearly teaches that when the object was updated the Canvas class would call 
the updater of that particular Component object within the model. Updating 
and/or drawing changes would proceed in the manner set forth in Seidl 8:25-65. 
The changed object would then be drawn, as set forth in 8:45-67. It would be 
obvious to perform this limitation in this manner, e.g. to modify Kinoe, for the 
reasons set forth in the rejection to claim 1 , which is incorporated by reference. 

As to claim 6, a method according to claim 2 wherein providing the 
selection graphic data comprises copying from the base graphic data non- 
selected objects that overlaps the selected graphic object. Kinoe and Seidl do 
not expressly teach this limitation while Kang does. Firstly, the rejection to claim 
4 is incorporated by reference in its entirety. As noted therein, occlusion 
processing is performed on graphical objects. Now, this occlusion processing of 
Kang that is well known in the art is performed during the rendering process. 
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Therefore, in order to perform occlusion processing, it would be obvious to one of 
ordinary skill in the art at the time the invention was made to copy objects that 
overlap with the selected object in order to perform occlusion processing for the 
reasons set forth in the rejection to claim 4 above. 

As to claims 7 and 14, wherein the highlighting attribute comprises a color 
attribute. Clearly, Kinoe teaches that graphical objects have attributes, including 
highlighting and transparency, as established above. As is clearly shown in Fig. 
13, no parts are highlighted, versus in Fig. 14 one part is highlighted, and it is 
very clear that the highlighting involves a change in color of the part from these 
drawings. 

As to claim 8, Kinoe teaches: wherein providing the selection graphic data 
comprises assigning a highlighting attribute to the copied selected graphic object. 
Kinoe clearly teaches this limitation in 2:52-60 and 3:1 -28, where it is taught that 
objects are structured in a hierarchy on the display, and that such objects can 
have various attributes and uses the term 'translucent' to mean 'transparent' in 
certain contexts, for example in 9:57-10:30 it is clearly taught that the graphical 
objects can be in highlighted, translucent, and/or translucent highlighted modes. 
Clearly, this means that each object has an attribute to indicate whether or not it 
is highlighted. 

As to claim 9, A method according to claim 8 wherein providing the 
selection graphic data comprises assigning a blank attribute to the copied non- 
selected objects. Kinoe teaches very clearly that the "highlighting attribute" 217 in 
graphical object state table 109 in Fig. 4 has a state of '0' or T, that is, an object 
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is either designated as highlighted ('1') or not ('0') - see 10:5-35. Objects by 
default are not highlighted (see for example Fig. 13). User selection and/or 
action result in highlighting attributes being enabled on specific parts (see Figs. 
14 and 15 for example). As stated in the rejection to claim 1 , which is 
incorporated by reference, the objects are copied in order to perform occlusion 
processing. Therefore, It would have been obvious to one of ordinary skill in the 
art at the time the invention was made that since objects by default are in a non- 
highlighted state, and that only the object that the user selects is highlighted, that 
such non-selected objects would have a blank attribute assigned to them, where 
the specification defines 'blank attribute' to mean 'non-highlighted.' 

As to claim 10, A method according to claim 9 wherein rasterizing the 
selection graphic data comprises assigning highlighting values to pixels 
associated with objects having highlighting attributes and assigning non- 
highlighting values to pixels associated with objects having blank attributes. 
Graphical objects prima facie consist of pixels. Therefore, any objects that are 
assigned highlighting attributes will prima facie have their pixel assigned 
highlighting values, and likewise pixels in objects that have a highlighting 
attribute of '0' will be assigned non-highlighting (e.g. normal) values (Kinoe 1 0:5- 
35). Clearly, the objects are rasterized before they are sent to the display device, 
and clearly the graphical object properties are taken into account when the object 
is rasterized, and Kang teaches that objects are processed in such a way that 
rasterizing occurs after altering the various attributes to the desired end state, 
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which has to occur because of the Seidl discussion presented in the rejection to 
claim 1 in any case. 

As to claim 12, a method according to claim 9 wherein the highlighting 
attribute and the blank attribute each comprise different color attributes. Kinoe 
clearly teaches this limitation as set forth in claim 8. Clearly the highlighted 
attribute is the dark color in Fig. 14 and the blank attribute is the light color, and 
clearly as taught by Kinoe as set forth in the rejection to claim 9 above, the state 
of the object when it is highlighted and not highlighted is different. Clearly, the 
difference in color as shown in Figs. 13 and 14 is sufficient proof that the color 
attributes are different. 

As to claim 49, wherein rasterizing the selection graphic data comprises 
assigning highlighting values to pixels in an area of the selection graphic raster 
corresponding to the copied selected graphic object and compositing the base 
graphic raster and the selection graphic raster comprises patterning areas within 
the output graphic raster corresponding to the area of the selection graphic 
raster. The above claim recites the same limitations as covered in the rejections 
to claims 3 and 4, which are incorporated by reference while not including the 
occlusion or overlap portions of claim 4. The last limitation, "patterning areas" is 
being interpreted using the broadest reasonable interpretation, where 
"patterning" is taken to mean applying some kind of image, texture, color, etc., to 
a graphical object. Further it would be obvious that if a user could apply a color 
to highlight an object, a user could also apply a pattern (texture), which is well 
known in the art (Examiner takes Official Notice that such use of other 
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highlighting schema (e.g. different pattern, texture, image, etc) are well known in 
the art at and that the reason for doing so is that different types of highlighting 
may be more appropriate to different circumstances, serve as better identifiers 
within a cluttered environment, allow color-blind users to distinguish highlighting, 
and other motivations). Motivation and combination is taken from the rejections 
to claims 3 and 4 above. 

Claims 17-20 are rejected under 35 U.S.C. 103(a) as unpatentable 
over Kinoe and Seidl as applied to claim 2 above, and further in view of 
Iwema et al (US PGPub 2003/0214539 A1). 

As to claim 1 7, comprising simplifying the copied selected graphic object. 
References Kinoe and Seidl do not expressly teach this limitation. Reference 
Iwema teaches a method of highlighting objects, wherein such objects are 
simplified, as in paragraph [0053], and the graphics of the objects are simplified, 
as in the last part of the paragraph, where alternative embodiments of the 
graphical objects can simply change the object to a predetermined color. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the Kinoe and Seidl references with Iwema as 
above for the reason that Iwema teaches improved methods for highlighting 
objects, and further teaches that simplifying graphical objects make it easier for 
them to be processed, e.g. requires less resources. 

As to claim 18, a method according to claim 17 where simplifying the 
selected graphic object comprises setting a plurality of color attributes of the 
selected graphic object to specify one color. Reference Iwema clearly teaches 
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this limitation in [0053], where it is set forth that the object could be made 
transparent, thusly have a background of red with white stripes (a plurality of 
color attributes) or other configuration, and that it is then set to a single 
predetermined color, which clearly meets the limitations of the recited claim, and 
motivation comes from the claim 1 7. 

As to claims 19 and 20, a method according to claim 17 wherein an 
exposed portion of the selected graphic object has an outline and simplifying the 
selected graphic object comprises replacing the selected graphic object with a 
shape bounded by the outline. Clearly, any object shown on the screen will have 
an outline (e.g. a graphical object must prima facie have bounds upon it so that it 
can be properly drawn or rendered (see Kinoe for this)), but Kinoe/Seidl do not 
expressly show this. The system of Iwema clearly teaches that objects can be 
surrounded with a halo around their outline in [001 1 -0015]. Further, it is taught 
that the body of the object can be turned into a predetermined background color, 
which would imply that the object would be replaced by a shape with that the 
color - or the bottom window shown in Fig. 2 and designated with cross marks 
through it. Therefore, for the reasons stated above with reference to claim 1 7 
cited from the Iwema reference, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify Kinoe/Seidl in the 
manner suggested therein as above, with motivation and rationale taken from the 
rejection to claim 1 7, which is incorporated by reference. 
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Claims 21, 23-26 and 39 are rejected under 35 USC 103(a) as 
unpatentable over Kinoe in view of Seidl as applied to claim 21 above, and 
further in view of Priem. 

As to claim 23, A method according to claim 21 wherein altering values of 
pixels from the base graphic raster comprises, for each pixel to be altered, 
computing a function to modify the value of the pixel to be altered, the function 
based on at least one of: the value of the pixel to be altered and the highlighting 
value of the corresponding pixel in the selection graphic raster. Reference Kinoe 
teaches that each graphical object is defined by a function (see Fig. 3, object 
definition data 203). Reference Priem teaches that the raster planes are 
combined using Boolean operations - see Priem 1 :5-21 and the abstract for 
example. It is well known in the art to use Boolean functions to combine raster 
objects (5:1 7-25). Therefore, it would have been obvious to one for ordinary 
skill in the art at the time the invention was made to modify Kinoe/Seidl to 
function in the manner recited in the Priem reference because it adds a more 
• flexible compositing operation. 

As to claim 24, a method according to claim 23 wherein the function 
comprises color inversion of the value of the pixel to be altered. Kinoe and Seidl 
do not expressly teach this limitation, but Priem further teaches that objects have 
foreground and background colors (see abstract) and that drawing planes can be 
drawn or painted in an inverted (e.g. color-inverted) mode using a Boolean 
operation (see for example Table 1 , 3:35-50), as separate from the 'draw 
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reversed' command), which clearly meets the limitations of this claim. Motivation 
and rationale are taken from the rejection to claim 23 above. 

As to claim 39, Seidl teaches this limitation (15:45-50, 64-65) as rendering 
means. 

As to claim 25, a method according to claim 23 wherein the function 
comprises performing one of a plurality of available color modification operations 
and wherein computing the function to modify the value of the pixel to be altered 
comprises selecting one of the plurality of available color modification operations 
based on the highlighting value of the corresponding pixel in the selection 
graphic raster and applying the selected color modification operation to the value 
of the pixel to be altered. Kinoe / Seidl does not expressly teach this limitation, 
while Priem teaches a plurality of Boolean operations to be used upon raster 
drawing planes (see for example Abstract and Table 1 - 3:35-50). Drawing 
planes can be drawn or painted in an inverted (e.g. color-inverted) mode using a 
Boolean operation (see for example Table 1 , 3:35-50), e.g. to choose the color 
values and patterns that such highlighting and/or inversion will take and/or use, 
which would allow the user to more easily perceive highlighting and/or choose 
colors and/or patterns that the user(s) find aesthetically pleasing and/or easier to 
see and/or to use (for instance, if the user were colorblind, the use of colors to 
differentiate highlighting would not be very useful). 

As to claim 26, a method according to claim 25 wherein the plurality of 
color modification operations comprises one or more of: performing color 
inversion on the value of the pixel to be altered; increasing the value of the pixel 
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to be altered by a predetermined amount; decreasing the value of the pixel to be 
altered by a predetermined amount; and, setting the value of the pixel to be 
altered to a predetermined value. Kinoe/Seidl do not teach this limitation. Priem 
teaches color inversion on pixels to be altered (Table 1 , 3:35-50). Motivation and 
combination is taken from the rejection to the parent claim and from motivation in 
claim 24. 

As to claims 21 , firstly, Kinoe assigns highlighting values to graphical 
objects (see for example Fig. 4, the highlighting attribute 217 in graphical object 
state table 1 09), where a graphic object is prima facie composed of pixels, where 
clearly as set forth by Seidl the selected object would be in its own memory area 
(or selection raster) as set forth in the rejection to claim 1 as previously 
discussed, the rejection to which is incorporated by reference. 

That is, Kinoe teaches changing an attribute of an object (which would 
exist in the selection graphics raster) by assigning a highlight value to it. Seidl 
clearly teaches that when the object was updated the Canvas class would call 
the updater of that particular Component object within the model. Updating 
and/or drawing changes would proceed in the manner set forth in Seidl 8:25-65. 
The changed object would then be drawn, as set forth in 8:45-67. Motivation is 
taken from the rejection to claim 1 , which is incorporated by reference. 

Claim 28 is rejected under 35 U.S.C. 103(a) as unpatentable over 
Kinoe in view of Seidl and Priem as applied to claim 23 above, and further 
in view of Kang et al (US 6,266,068 B1). 
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As to claim 28, wherein rasterizing the selection graphic data to yield the 
selection graphic raster comprises assigning highlighting values only to pixels in 
the selection graphic raster corresponding to portions of the selected graphic 
object that are not overlapped by other non-transparent graphic objects. 
References Kinoe, Seidl, and Priem do not expressly teach this limitation. 
However, it is well known in the art that three-dimensional graphics rendering 
systems such as Kinoe (15:47-60) do not display parts that are hidden from view 
or the portions that overlap thereof. 

Kang et al (US 6,266,068 B1) clearly teaches the existence of multiple 
graphics planes in Figs. 2 and 3A-3E, and in 2:30-65, where as stated above, 
one of those layers could be the base graphics raster. Kang clearly teaches that 
users can select objects; see for example the intra-layer selector 106 in Fig. 1 as 
taught in 4:19-35. Kang teaches in 1 1 :37-63 that a user can highlight a layer 
using the input device to select the layer in question, with each layer having a 
different depth (e.g. see for example Figs. 3A-3E with emphasis on Figs. 3C and 
3D with the shaded ellipse having different depths in each, see for example 8:22- 
46). 

Kang further teaches that his system can deal with complex occlusion 
scenarios (1 :38-61), and this is noted in 4:20-35, where layers that occlude or are 
blended are disclosed, and in 7:10-30 the use of a painter's algorithm is 
disclosed, where this algorithm is well known in the art to ensure that objects in 
the scene occlude each other correctly based on depth, except where such 
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objects are blended because they are partially transparent as set forth in 4:20-35 
for example. 

All that being said, it would be obvious to one of ordinary skill in the art at 
the time the invention was made that only the non-occluded portions of such an 
object would be drawn when a painter's algorithm or similar is used so that the 
scene is drawn in a realistic manner (excluding pixels that are blended as set 
forth above). Therefore, since only the portions of the selected graphical object 
that are not occluded will be drawn, (requires less system resources and is 
therefore inherently faster, more efficient, and requires less powerful hardware to 
implement than its lack.) Therefore, It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to only highlight the 
portions that are visible, as the rest of the object will not be drawn anyway. 

Claim 45 is rejected under 35 U.S.C. 103(a) as unpatentable over 
Kinoe / Seidl as applied to claim 1 above, and further in view of Good et al 
(US PGPub 2003/0156124). 

As to claim 45, It would have been obvious to one of ordinary skill in the 
art at the time the invention was made to combine Good with the invention of 
Kinoe / Seidl, because Good teaches methods of highlighting objects that include 
flashing - [0076], which would allow the user to more easily perceive highlighting 
and/or choose colors and/or patterns that the user(s) find aesthetically pleasing 
and/or easier to see and/or to use (for instance, if the user were colorblind, the 
use of colors to differentiate highlighting would not be very useful). See MPEP 
2144.04(1). 
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Claims 29-30 and 40-41 are rejected under 35 U.S.C. 103(a) as 
unpatentable over Kinoe / Seidl in view of Priem as applied to claim 23 
above, and further in view of US 5,438,552 to Audi et al. 

As to claim 29, A method according to claim 23 wherein compositing the 
base graphic raster and the selection graphic raster comprises identifying 
contiguous regions of pixels in the base graphic raster where corresponding 
pixels in the selection graphic raster have highlighting values. References 
Kinoe/Seidl do not teach this limitation expressly. Reference Audi teaches a 
method for identifying regions of contiguous pixels that are the same such that 
they can be displayed and separated from the rest of the base graphic regions 
(background)-see Figures 9 et al, and 8:55-9:65. This would be effective for the 
case where there are multiple parts or objects in the base graphics raster that are 
highlighted and that need to be processed; such a technique would be fast and 
allow easy segmentation of the elements to be easily matched with the objects in 
the selection graphics raster or layer. It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to combine the 
systems of Kinoe/Seidl with that of Audi utilizing the contiguous region extraction 
method of Priem for the reason that Priem allows easy extraction of contiguous 
regions 8:55-9:55, where it is taught that such methods can segment out 
foreground image regions, which correspond to the foreground colors or images 
of Priem (1 :5-25) for example. 

As to claim 40, K/L/S do not expressly teach this limitation, but Reference 
Audi teaches that a processor (10:44-45) uses an algorithm (9:1-12:63) to 
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identify contiguous regions of pixels (8:53-9:20) and put them in one layer and 
then put the rest of the pixels in another layer (Figure 9a), e.g. a method for 
identifying contiguous regions of pixels in the base graphics as recited in present 
limitation. Motivation is taken from the rejection to claim 39 above. 

As to claims 30 and 41 , A method according to claim 29 wherein altering 
values of pixels from the base graphic raster comprises altering values of a 
pattern of selected pixels in the contiguous regions of pixels in the base graphic 
raster. Kinoe and Seidl-Priem do not expressly teach this limitation, but Audi 
clearly teaches highlighting different regions with different colors. Therefore, it 
would be obvious to allow the user the choice of color or pattern to apply to 
illustrate highlighting. See MPEP 2144.04(1). 

Claims 34-37 are is rejected under 35 U.S.C. 103(a) as unpatentable 
over Kinoe in view of Seidl as applied to claim 1 above, and further in view 
of Goodwin et al (US 5,818,975). 

As to claim 34, the rejection to claim 1 is incorporated by reference. The 
additional limitation of deleting non-selected objects is taught by Goodwin as in 
Figures 5 and 6, where regions that were NOT R2 or NOT R1 are deleted from 
the respective images, and replaced with an average R2 or R1 value. Clearly, 
the idea of deleting objects that do not meet specified criteria is taught by 
Goodwin, and it would be obvious that if non-highlighted portions had a different 
dynamic range, they would be deleted. It would be obvious that one layer 
containing foreground objects could extract those alone simply by applying an 
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OR operation to the foreground and background raster planes, which would be of 
great use in techniques such as chroma-keying, which are well known in the art. 

Motivation to combine is taken from the fact that Goodwin would allow the 
system of Kinoe and Seidl so that it could support high dynamic range images 
that had a higher dynamic range than the monitor by allowing the user to select 
desired regions of one specificity such that they would be deleted, for example in 
cases with high-dynamic range images from professional-grade digital cameras. 

As to claim 35, Kinoe teaches: wherein providing the selection graphic 
data comprises assigning a highlighting attribute to the copied selected graphic 
object. Kinoe clearly teaches this limitation in 2:52-60 and 3:1-28, where it is 
taught that objects are structured in a hierarchy on the display, and that such 
objects can have various attributes and uses the term 'translucent' to mean 
'transparent* in certain contexts, for example in 9:57-10:30 it is clearly taught that 
the graphical objects can be in highlighted, translucent, and/or translucent 
highlighted modes. Clearly, this means that each object has an attribute to 
indicate whether or not it is highlighted. Clearly such objects are copied. 

As to claim 36, this claim has the same rationale as the rejection to claim 
32, which is incorporated by reference. 

As to claim 37, clearly the highlighted attribute is the dark color in Fig. 14 
and the blank attribute is the light color, and clearly as taught by Kinoe as set 
forth in the rejection to claim 9 above, the state of the object when it is 
highlighted and not highlighted is different. Clearly, the difference in color as 
shown in Figs. 13 and 14 is sufficient proof that the color attributes are different. 
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Kinoe Figs. 13-15, both the highlighting attribute and the lack thereof are clearly 
color attributes, and in this case they are the opposite setting of each other. 

Claim 33 is rejected under 35 USC 103(a) as unpatentable over Kinoe, 
Seidl, and Goodwin as applied to claim 32 above, and further in view of 
Priem. 

As to claim 33, clearly as set forth in Kinoe, non-selected / highlighted 
parts are one color and highlighted/selected parts are another. However, color 
inversion perse as a means of highlighting is not taught in Kinoe/Seidl/Goodwin. 
Furthermore, if the highlighting comprised the inversion taught in Priem (for 
example, Table 1 , 3:35-50), then obviously that would constitute a color attribute 
since it obviously increases the contrast ratio. The instant specification defines 
'blank attribute" to mean 'non-highlighted.' [0064]. The objects of Kinoe have this 
highlighting attribute and maintain a highlighted or non-highlighted state. 

Claims 38 and 42-43 are rejected in view of Kinoe and Seidl as 
applied to claim 1, and further in view of Altman (US PGPub 2004/0163042 
A1, EFD 07/26/00). 

As to claim 38, Kinoe and Seidl do not expressly teach two files containing 
the base graphics and the highlighted objects. Altman teaches the use of a 
second file that contains highlighted objects [0045] that are then overlaid onto the 
first file and composited with it [0054, claim 1]. It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to modify Kinoe 
/ Seidl to use such functionality because it allows highlights and comments to be 
stored without affecting changes in the original base document / file / etc. [0045] 



Application/Control Number: 10/677,332 Page 
Art Unit: 2628 

As to claims 42, Kinoe/Seidl do not expressly teach but Altman teaches 
PDF files for such annotations and original base documents - see Figure 1 , 
[0005-0007, 0008-0009 specifically]. Motivation taken from the rejection to claim 
38, which is incorporated by reference. 

As to claim 43, examiner takes Official Notice that it would have been 
obvious to store such information in tags because it facilitates easy search and 
retrieval of the secondary documentation and because this is a standard format 
for storing textual information in other computer languages. 

Claims 46-47 are rejected under 35 U.S.C. 103(a) as unpatentable 
over Kinoe in view of Seidl as set forth in the rejection to claim 1 above, 
and further in view of Spriggs et al (US 6,421,571 B1). 

As to claim 46, A method according to claim 1 wherein the base graphic 
raster includes a plurality of selected graphic objects to be highlighted, the 
plurality of selected graphic objects including at least graphic objects of first and 
second types and wherein providing the selection graphic data comprises 
providing in the selection graphic data an object corresponding to each of the 
plurality of selected graphic objects and assigning a highlighting attribute to each 
of the objects, the method comprising assigning different highlighting attributes to 
objects corresponding to graphic objects of the first and second types. Kinoe 
and Seidl do not expressly teach this limitation. Reference Spriggs is directed to 
the same problem-solving area, that is the effective communication of information 
concerning graphical objects using highlighting. Further, in claims 9-1 1 , Spriggs 
teaches that various graphical objects of different types are shown on the screen 
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in a hierarchical fashion, and further that such objects are known to have 
different colors assigned to them based on their state, and that the use can 
highlight them. In light of Spriggs and MPEP 2144, and the above reasons, it 
would have been obvious to one of ordinary skill at the time the invention was 
made to combine the systems of Kinoe and Seidl with Spriggs, as the system of 
Spriggs would allow the user to highlight different types of graphical objects and 
'drill down' to get to the desired part or object, and to do so with two or more 
different types of part present in one scenario, whereas in Kinoe only one type of 
object can be highlighted per pass and clearly distinguished, since only one color 
is utilized. 

As to claim 47, clearly if one object can be copied, a plurality of objects 
can be copied, where it would be obvious to have a plurality of selected graphic 
objects if one object exists. Motivation incorporated from the parent rejection, 
which is incorporated by reference 

Claim 48 is rejected under 35 U.S.C. 103(a) as unpatentable over 
Kinoe, Seidl, and Spriggs as applied to claim 47 above, and further in view 
of Iwema. 

As to claim 48, a method according to claim 47 wherein copying the 
plurality of selected graphic objects comprises simplifying one or more of the 
plurality of selected graphic objects. Clearly, any object shown on the screen will 
have an outline (e.g. a graphical object must prima facie have bounds upon it so 
that it can be properly drawn or rendered (see Kinoe for this)), but Kinoe/Seidl do 
not expressly show this. The system of Iwema clearly teaches that objects can 
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be surrounded with a halo around their outline in [001 1-0015]. Further, it is 
taught that the body of the object can be turned into a predetermined background 
color, which would imply that the object would be replaced by a shape with that 
the color - or the bottom window shown in Fig. 2 and designated with cross 
marks through it. Therefore, for the reasons stated above with reference to claim 
1 7 cited from the Iwema reference, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify Kinoe/Seidl in the 
manner suggested therein as above. 

Claim 50 is rejected under 35 U.S.C.103 (a) as obvious over Kinoe, 
Seidl, and Kang in view of AAPA. (Applicant did not challenge this taking 
of Official Notice in the last Office Action; therefore it stands as applicant 
admitted prior art (AAPA)). 

As to claim 50, A method according to claim 49 comprising creating a 
plurality of output graphic rasters, for each of the plurality of output graphic 
rasters differently patterning the areas within the output graphic raster, and 
displaying the plurality of output graphic rasters in rotation. The references 
applied to claim 49 do not expressly teach this limitation. However, it is well 
known in the art of web design to have animated GIF (BMP, JPEG) images, 
utilizing the HTML 4.0.1 standard tag <img alt="animated eyes" src="eyes.gif" 
border="0" width="24" height="12"> as an example. It would have been obvious 
to combine Kinoe/Seidl/Kang and AAPA to use this system for displaying 
graphics from the web, e.g. showing web page and/or using the system for 
graphic WYSIWYG layout of webpages and/or graphics overall. 
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Claim 51 is rejected under 35 U.S.C. 103(a) in view of Kinoe and Seidl 
in view of Iwema. 

As to claim 51 , A method according to claim 1 wherein the base graphic 
raster has a higher resolution than the selection graphic raster. Kinoe and Seidl 
do not expressly teach this limitation while Iwema in [0078] teaches that in some 
embodiments that highlighting halo may be anti-aliased for improved resolution, 
which clearly requires that the halo be composited at a different resolution than 
the monitor (e.g. super-sampling). Iwema teaches a glow / halo around a 
highlighted object that makes it more noticeable. This effect would clearly be 
useful in the system of Kinoe and make highlighted parts more visible. [001 1 - 
0015], Fig. 2, as with designated with cross marks through it. The combination 
would thusly have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kinoe and Seidl to use the halo effect of Iwema for 
the reasons above. 

Conclusion 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Eric Woods whose telephone number is 571 - 
272-7775. The examiner can normally be reached on M-F 7:30-5:00. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Ulka Chauhan can be reached on 571-272-7782. The fax 
phone number for the organization where this application or proceeding is 
assigned is 571 -273-8300. 
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Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
9199 (IN USA OR CANADA) or 571-272-1000. 

Eric Woods May 25, 2007 
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